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ABSTRACT 


It  is  essential  for  the  Marine  Corps  to  ensure  the  successful  supply,  movement  and 
maintenance  of  an  armed  force  in  peacetime  and  combat.  Integral  to  an  effective, 
long-term  logistics  plan  is  the  ability  to  accurately  forecast  future  requirements  to  sustain 
materiel  readiness.  Total  Life  Cycle  Management  -  Assessment  Tool  (TLCM-AT)  is  a 
simulation  tool  combining  operations,  maintenance,  and  logistics.  This  exploratory 
analysis  gives  insight  into  the  factors  used  by  TLCM-AT  beyond  the  tool’s  embedded 
analytical  utilities.  A  Java  program  is  developed  to  automate  multiple  changes  to 
TLCM-AT ’s  database,  execute  simulation  runs  and  record  output  data.  A  scenario 
deploying  LAV-25  vehicles  to  a  tropical  region,  with  three  courses  of  action,  provides  the 
basis  for  analysis.  The  research  provides  a  description  of  the  analysis  available  by 
TLCM-AT  as  a  stand-alone  tool,  and  concludes  with  how  design  of  experiments  (DOE) 
expands  insights  gained.  This  thesis  provides  a  framework  for  using  DOE  with 
TLCM-AT,  identifies  a  structured  use  of  TLCM-AT  for  decision  makers,  and  provides 
enhancements  that  enable  more  effective  use  of  TLCM-AT.  Results  indicate  no  practical 
change  in  operational  availability  (Ao)  when  varying  five  factors,  using  129  design  points 
and  15,480  replications.  The  factors  adjusted  are:  spares,  depot  capacity,  induction 
quantity,  part  repair  time  and  part  degradation  time.  Results  also  reveal  synergies 
between  the  modeled  factors  and  numbers  of  spares  to  be  the  dominant  factor  Ao. 
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EXECUTIVE  SUMMARY 


Logistics  is  a  vital  part  of  the  Marine  Corps’  vision  of  developing  a  force  capable 
of  performing  and  successfully  completing  the  missions  of  the  twenty-first  century.  It  is 
essential  that  the  Marine  Corps  ensures  the  successful  supply,  movement  and 
maintenance  of  an  armed  force  in  peacetime  and  combat. 

This  thesis  centers  on  the  Total  Life  Cycle  Management  -  Assessment  Tool 
(TLCM-AT).  Total  Life  Cycle  Management  (TLCM)  defines  the  design,  engineering, 
manufacturing,  disassembly  and  disposal  of  a  system.  Decision  makers  who  use  TLCM 
must  understand  its  impact  on  combat  effectiveness  and  on  the  development  of  systems 
required  for  success  in  modem  warfare.  The  Marines  acquired  TLCM-AT  to  improve 
upon  their  ability  to  manage  TLCM  and  maintain  a  more  reliable  fighting  force.  This 
summary  provides  a  functional  overview  of  TLCM-AT,  the  research  methodology, 
conclusions  and  recommendations.  The  analysis  is  designed  to  explore  the  capabilities  of 
TLCM-AT  with  multiple  quantitative  techniques,  in  order  to  develop  an  analytical 
process  that  enables  tactical  users  to  more  readily  employ  TLCM-AT. 

TLCM-AT  is  a  simulation  tool  developed  by  Clockwork  Solutions  in  Austin, 
Texas.  It  is  a  holistic,  continuous-loop  representation  of  the  life  cycle  of  a  weapon 
system:  combining  operations,  maintenance,  and  logistics.  TLCM-AT’s  model  structure 
and  organization  enables  the  Marine  Corps  to  model  the  myriad  of 
industry-accepted  elements  that  directly  affect  the  life  cycle  of  a  system.  TLCM-AT 
assists  weapons  systems  managers  with  evaluating,  quantifying,  and  reducing  life-cycle 
costs  without  adversely  impacting  fleet  readiness  and  availability.  Users  can  run  “what 
if’  scenarios  by  manipulating  data  inputs  to  examine  the  long-term  effects  to  life-cycle 
policy  decisions.  Scenarios  are  adjusted  by  the  user  via  the  scenario  editor  in  TLCM- 
AT’s  graphical  user  interface  (GUI).  This  research  extends  the  nonnal  analysis  by 
implementing  the  concepts  of  data  farming  and  design  of  experiments  (DOE)  as 
employed  by  the  Simulation  Experiments  and  Efficient  Designs  (SEED)  Center  at  the 
Naval  Postgraduate  School  (NPS). 
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This  thesis  provides  the  following: 


■  A  framework  for  using  DOE  with  TLCM-AT. 

■  A  structured  use  of  TLCM-AT  for  USMC  decision  makers. 

■  Enhancements  that  enable  more  effective  usage  of  TLCM-AT. 

A  notional  scenario  deploying  Light  Annored  Vehicles  (LAVs)  to  a  tropical 
environment  serves  as  the  background  for  this  prototype  analysis.  According  to  the 
scenario,  the  Marine  Corps  seeks  to  deploy  one  LAV-25  battalion,  and  eventually  a 
follow-on  battalion  for  contingency  operations  in  a  hot,  humid  environment. 
Maintenance  personnel  anticipate  numerous  unscheduled  removal  events  of  two 
computerized  parts  that  historically  perform  poorly  in  humid  conditions.  Three  potential 
courses  of  action  (COAs)  are  suggested  to  help  mitigate  the  anticipated  problem. 

COA  1:  Send  a  large  number  of  spares  with  the  follow-on  battalion. 

COA  2:  Invest  in  improved  Line  Replacement  Units  (LRU).  The  research 

and  development  (R&D)  cost  for  the  one-month  R&D  program  is 
$1M,  and  the  new  LRUs  cost  an  additional  1.5  times  the  cost  of  the 
old  type.  Bring  a  lesser  number  of  the  improved  spares  and  install 
them  whenever  the  old  type  is  removed.  When  the  old  type  is 
condemned  (removed  from  maintenance  system),  buy  the 
new  type. 

COA  3:  A  variant  of  COA  2,  except  that  the  old  type  is  purchased 

whenever  the  LRUs  are  condemned.  No  money  is  invested  in  the 
new  LRUs.  The  idea  is  to  save  some  money,  while  accomplishing 
the  same  goals. 

Each  scenario  is  modeled  separately  in  TLCM-AT’s  Microsoft  Access  database. 
Five  factors  are  chosen  to  explore  over  the  COAs  using  Nearly  Orthogonal  Latin 
Hypercube  (NOLH)  experimental  designs.  A  Java  program  is  created  to  automatically 
run  each  scenario  using  129  input  combinations.  The  five  factors  varied  in  this  study  are: 

■  Spare  levels:  The  total  number  of  spares  at  each  repair  location. 

■  Induction  Quantity:  A  limit  on  the  number  of  inductions  (number  of 
items  a  maintenance  facility  can  accept)  that  can  occur  in  the  given  quarter 
and  year  at  a  single  repair  facility. 

■  Capacity:  The  number  of  parts  that  can  be  processed  concurrently  at  a 
single  repair  facility. 
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Service  Times:  The  time  it  takes  to  repair  a  part. 
Unscheduled  Removal  Rates:  The  part  failure  rate. 


The  NOLH  allows  for  systematic  variation  of  the  factors  to  broadly  explore  their 
effect  on  the  measure  of  effectiveness  (MOE).  Ao  is  chosen  as  the  MOE  for  this  study. 
Upon  completion  of  the  Java  program,  a  comma-separated  file  is  created  containing  Ao 
by  quarter  over  a  20-quarter  period. 

First,  each  scenario  is  run  once  with  100  replications.  This  type  of  simulation 
typifies  the  kind  of  study  a  common  user  of  TLCM-AT  would  use  to  produce  a 
quick-turn  analysis  of  a  specific  scenario.  Results  show  COA-2  saves  $3.2M  compared 
to  COA-1  and  $15.8M  compared  to  COA-3.  COA-2  had  lower  overall  operation  and 
sustainment  (O&S)  costs  for  spares  replenishment  and  produced  the  least  impact  on 
repair  facilities  by  producing  a  lower  number  of  maintenance  tasks  perfonned. 
Furthermore,  each  scenario  maintained  relatively  equal  Ao  throughout  the  contingency 
and  for  the  full  20-quarter  period.  All  scenarios  produced  a  much  higher  Ao  than  the 
baseline,  thus  maintaining  the  status  quo. 

DOE  implementation  with  TLCM-AT  across  all  scenarios  and  129  design  points 
each  produced  similar  results  in  regard  to  Ao.  Statistically  significant  differences  in  Ao 
are  observed  after  quarter  12  (the  end  of  contingency  operations).  However,  these 
differences  are  not  practically  significant,  as  Ao  varied  no  more  than  4%  for 
each  scenario. 

Regression  analysis  produced  relatively  low  R2  values,  representing  a  low  amount 
of  variance  explained  in  the  model  by  the  five  factors.  In  cases  where  R"  is  low,  the 
results  show  that  other  factors  (besides  those  used  in  this  study)  should  be  considered. 
Thirty-two  stepwise  regressions  are  conducted,  and  R"  increases  in  all  cases  as  interaction 
tenns  are  added  to  the  models.  This  result  shows  that  including  main  effects  alone  did 
not  capture  the  complete  picture.  Significant  factors  for  scenario  two  are  determined  to 
be  spares,  spares  x  spares,  spares  x  service  time  and  induction  quantity  x  degradation 
time.  The  significance  of  interaction  terms  reveals  synergies  between  the  factors. 

Paired  two-sample  t-Tests  between  each  scenario  are  completed  to  detennine 
whether  or  not  two  population  means  of  Ao  are  equal.  In  each  test,  the  hypothesized 
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mean  difference  is  zero.  Low  p-values  are  produced  for  all  six  comparisons,  revealing  a 
statistically  significant  difference  in  mean  Ao  between  each  scenario.  However,  the 
mean  difference  is  not  practically  significant  because  the  variation  in  mean  Ao  is  small. 
For  example,  going  from  81.8%  to  82.1%  Ao  is  not  significant  enough  to  make  practical 
impacts  on  the  overall  operational  picture  over  a  range  of  inputs.  However,  there  are 
some  differences  in  other  measures  such  as  cost  and  maintenance  utilization. 

In  conclusion,  exploitation  of  TLCM-AT  is  accomplished  through  development  of 
a  Java  program  to  automate  implementation  of  DOE  and  NOLH  with  TLCM-AT.  The 
simulated  scenarios  used  in  this  analysis  fonn  a  strong  foundation  for  further  TLCM-AT 
studies  that  use  DOE  to  analyze  life  cycles.  Java  code  written  to  extract  many  different 
MOEs  gives  insight  to  the  factors  most  affecting  outcomes  of  simulation  runs.  This 
analysis  adds  depth  to  typical  “what  if’  scenario  runs,  and  informs  decision  makers  on 
the  consequences  of  decisions  regarding  Marine  Corps  weapon  systems.  It  is 
recommended  that  research  continue  into  embedding  TLCM-AT  in  a  data  fanning 
environment.  Furthermore,  it  is  recommended  that  a  prototype  study  be  performed. 
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I.  INTRODUCTION 


Develop  better  readiness  and  sustainment  indicators  based  on  predictive 
modeling,  so  that  timely  changes  to  strategies,  plans,  and  programs  can 
be  implemented. 


Commandant’s  Planning  Guidance,  November  2006 

A.  BACKGROUND  AND  MOTIVATION 

The  United  States  Marine  Corps  (USMC)  continually  works  to  shape  logistics 
plans  and  policies  to  sustain  excellence  in  combat  effectiveness.  Logistics  is  an 
extremely  vital  part  of  the  Marine  Corps’  vision  of  developing  a  force  capable  of 
performing  and  successfully  completing  the  missions  of  the  twenty-first  century.  It  is 
essential  that  the  Marine  Corps  ensures  the  successful  supply,  movement  and 
maintenance  of  an  armed  force,  both  in  peacetime  and  in  combat.  Integral  to  an  effective, 
long-tenn,  logistics  plan  is  the  ability  to  accurately  forecast  future  requirements  to  sustain 
materiel  readiness  as  well  as  detennine  future  force  structural  needs.  This  study  focuses 
on  the  elements  that  affect  the  life  cycle  of  weapons  systems,  such  as  level  of  spares, 
depot  capacity  and  repair  times. 

Enterprise-level  Total  Life  Cycle  Management  (TLCM)  is  the  formal  process  to 
identify,  analyze,  and  implement  synergistic  “cradle-to-grave”  solutions  that  optimize  the 
acquisition  and  logistics  chain  across  the  Marine  Corps.  The  following  items  impact  the 
entire  Marine  Corps  and  require  an  enterprise  view1: 

■  Automatic  Logistics/Prognostics 

■  Performance-Based  Logistics 

■  Design-in  Reliability,  Maintainability  and  Supportability 

■  Direct  Vendor  Delivery 

■  Logistics  Footprint 

■  Fuel  Efficiency 


1  Marine  Corps  Order  4000.57,  Enclosure  (1),  Commandant  of  the  Marine  Corps,  Subject:  Marine 
Corps  Life  Cycle  Management,  16  September  2005,  p.  1. 
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■  Depot  Maintenance 

■  Condition-Based  Maintenance  Technology 

■  Logistics  Operational  Architecture 

■  Automated  Identification  Technology 

TLCM  defines  the  design,  engineering,  manufacturing,  disassembly  and  disposal 
of  a  system.  TLCM  includes  the  set  of  decisions  and  actions  that  determine  the 
performance  and  availability  of  a  weapons  system  in  the  context  of  its  environment. 
Therefore,  decision  makers  must  use  TLCM  and  understand  its  impact  on  combat 
effectiveness  and  on  the  development  and  cost  of  weapons  systems  needed  for  success  in 
modem  warfare. 

The  Marine  Corps  has  invested  in  Clockwork  Solutions  (www. clockwork- 
solutions. com)  to  adapt  a  discrete  event  simulation  tool  for  use  in  conducting  TLCM. 
Clockwork  Solutions  is  a  provider  of  reliability-centered  total  life  cycle  performance 
prediction  tools.  One  of  their  simulation  tools,  the  TLCM  Assessment  Tool  (AT),  is  an 
emerging  enhancement  module  to  the  Life  Cycle  Modeling  Integrator  (LCMI).  LCMI  is 
a  Web-based  set  of  decision  support  modules  that  integrates  historical  data  and  converts 
it  to  quality  logistics  intelligence  for  the  USMC.  Within  the  TLCM-AT  module  is  the 
Fault  Isolation  (FI)  model,  which  uses  a  multivariate,  dimension  reduction  algorithm  to 
create  a  ranked  index  of  the  components  of  a  principal  end  item  (PEI).  Program 
Managers  (PMs)  can  use  this  decision  support  tool  to  enhance  the  effectiveness  of  their 
depot  maintenance  rotation  programs.  The  FI  model  can  also  help  identify  components 
to  be  inspected  during  Inspect  and  Repair  Only  As  Necessary  (IROAN)  programs. 

The  aging  of  such  systems  as  the  Amphibious  Assault  Vehicle  (AAV)  and  the 
Light  Armored  Vehicle  (LAV)  creates  a  greater  level  of  uncertainty  about  their  Ao. 
Simulation  tools,  such  as  TLCM-AT,  are  helpful  in  identifying  critical  factors  that  affect 
the  Ao  of  weapon  systems.  Simulation  can  make  forecasting  more  accurate  and  enable 
decision  makers  to  be  proactive  with  TLCM.  An  exploratory  analysis  on  model  factors 
pinpoints  factors  with  the  strongest  effects  on  output  such  as  system  reliability,  Ao  and 
cost.  Furthermore,  sensitivity  analysis  of  critical  factors  better  defines  thresholds  for 
decision  makers.  This  thesis  examines  the  capabilities  of  TLCM-AT. 
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B.  RESEARCH  QUESTIONS 


The  exploratory  analysis  gives  insight  into  the  factors  used  by  TLCM-AT.  While 
this  analysis  is  not  exhaustive,  the  following  questions  guide  this  research: 

■  Which  factors  modeled  in  the  TLCM-AT  have  a  critical  effect  on  the  Ao 
of  the  LAV-25? 

■  How  should  data  farming  be  applied  to  the  TLCM-AT  model? 

■  How  sensitive  are  the  factors  (spares,  capacity,  induction,  service  time, 
degradation  time)  to  defining  thresholds  for  decision  makers? 

C.  SCOPE  OF  THESIS 

This  analysis  examines  the  capabilities  of  the  TLCM-AT.  The  goal  is  to  use 
multiple  quantitative  analysis  techniques  in  order  to  develop  a  methodology  to  enable 
tactical  users  to  more  readily  employ  the  TLCM-AT  tool. 

D.  METHODOLOGY 

This  thesis  uses  the  TLCM-AT,  for  which  three  scenarios  are  created,  each 
revolving  around  a  potential  decision  about  the  LAV-25.  Each  scenario  represents  a 
possible  course  of  action  (COA)  for  the  decision  maker.  In  each  scenario,  five  factors 
are  adjusted: 

■  Spare  levels:  The  total  number  of  spares  at  each  repair  location. 

■  Induction  Quantity:  A  limit  on  the  number  of  inductions  that  can  occur 
in  the  given  quarter  and  year  at  a  single  repair  facility. 

■  Capacity:  The  number  of  parts  that  can  be  processed  concurrently  at  a 
single  repair  facility. 

■  Service  Times:  The  time  it  takes  to  repair  a  part. 

■  Unscheduled  Removal  Rates:  The  part  failure  rate. 

This  study  determines  the  effect  these  factors  have  on  the  Ao  of  the  LAV -25.  Ao 
is  defined  as  the  number  of  operational  platforms  divided  by  the  total  number  of 
platforms  at  a  given  base  or  bases  at  a  given  future  time  interval.  Each  scenario  is 
explained  in  detail  in  Chapter  III. 
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The  scenarios  are  developed  based  on  a  range  of  inputs  of  the  factors  listed  above. 
The  factors  are  used  to  create  several  design  points  developed  using  the 
Nearly  Orthogonal  Latin  Hypercube  (NOLH).2  Scenario  runs  are  replicated  in  the 
simulation  and  then  analyzed.  The  analysis  uses  data  farming,  as  employed  by  the 
Simulation  Experiments  and  Efficient  Designs  (SEED)  Center  at  the  Naval  Postgraduate 
School  (NPS)  and  the  International  Data  Fanning  Workshop.  Data  farming  uses 
high-speed  computing  to  run  simulations  thousands  of  times  while  simultaneously 
varying  many  input  factors.  Using  experimental  designs  developed  at  NPS,  the  data 
farming  results  is  analyzed.  The  statistical  analysis  reveals  the  strengths,  weaknesses  and 
sensitivities  of  the  model.  The  result  is  a  preliminary  study  of  support,  based  on  analysis, 
to  aid  future  users  of  TLCM-AT. 

E.  BENEFITS  OF  THE  STUDY 

This  study  provides  the  Marine  Corps  with  analytical  support  for  TLCM-AT. 
Data  farming  and  NOLH  provide  a  unique  approach  for  the  study  by  allowing  the 
identified  factors  to  be  widely  varied,  and  for  multiple  simulation  runs  to  be 
accomplished  for  post-run  analysis.  Chapter  II  provides  a  brief  description  of  discrete 
event  simulation  modeling  followed  by  a  functional  overview  of  TLCM-AT.  Chapter  III 
explains  specifically  how  data  fanning  and  NOLH  are  used  in  the  simulation  runs. 
Chapter  IV  contains  data  analysis  from  the  resulting  data  obtained  from  multiple 
simulation  runs.  Chapter  V  provides  conclusions  and  identifies  future  areas  of  study. 
Currently,  the  Marine  Corps  does  not  know  all  the  capabilities  and  limitations  of 
TLCM-AT.  The  aim  of  this  study  is  to  provide  those  insights  into  how  TLCM-AT  can  be 
used  to  help  decision  makers  make  more  informed  decisions  regarding  TLCM. 


-  NOLH  concept  developed  by  Lieutenant  Colonel  Thomas  M.  Cioppa,  United  States  Army.  Detailed 
description  found  in  his  dissertation,  “Efficient  Nearly  Orthogonal  and  Space-Filling  Experimental  Designs 
for  High-Dimensional  Complex  Models,”  Ph.D.  in  Operations  Research,  Naval  Postgraduate  School, 
Monterey,  California,  September  2002. 
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II.  MODEL  BACKGROUND  AND  CAPABILITIES 


A.  INTRODUCTION 

Before  conducting  an  exploratory  analysis,  one  must  understand  the  intricacies  of 
the  model  itself.  The  purpose  of  this  chapter  is  to  introduce  the  reader  to  TLCM-AT.  It 
begins  with  a  brief  description  of  mathematical  models,  both  detenninistic  and  stochastic. 
This  discussion  leads  into  a  functional  overview  of  TLCM-AT,  including  criteria, 
structure  and  applications. 

B.  DISCRETE  EVENT  STOCHASTIC  MODELING 

Mathematical  models  can  be  categorized  broadly  as  probabilistic  or  detenninistic. 
A  detenninistic  model  uses  factors  and  variables  without  random  fluctuations.  The 
system  is  entirely  defined  by  the  initial  conditions.  A  detenninistic  model  provides  a 
single  point  estimate  which  serves  as  a  “best  guess”  for  an  unknown  population 
parameter.  A  stochastic  model  considers  randomness  in  one  or  more  of  its  factors  or 
variables.  By  allowing  random  variation  in  one  or  more  inputs  over  time,  a  stochastic 
model  estimates  probability  distributions  of  potential  outcomes.  What  results  is  not  a 
single  point  estimate  but  a  distribution  of  possible  outcomes. 

After  empirical  data  is  collected,  a  theoretical  probability  distribution  is  often  fit 
to  the  data  to  attain  a  more  general  description  of  the  underlying  process.  The  population 
parameters  can  be  approximated  for  the  distribution.  Stochastic  processes  provide  a 
better  understanding  of  inherently  stochastic  real-life  situations. 

The  TLCM-AT  is  a  discrete  event  stochastic  model.  The  aim  of  TLCM-AT  is  the 
integration  of  discrete  event  perfonnance  models  into  the  USMC  decision  support 
enterprise.  Clockwork  Solutions  has  modeled  five  USMC  systems: 

■  Light  Armored  Vehicle  (LAV-25) 

■  Medium  Tactical  Vehicle  Replacement  (MTVR) 

■  Lightweight  1 55mm  Howitzer  (LW 155) 

■  Joint  Light  Tactical  Vehicle  (JLTV) 

■  Amphibious  Assault  Vehicle  (AAV) 
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These  five  systems  are  chosen  based  on  the  following  criteria: 

■  Maintenance  Cost 

■  Operational  Usage 

■  High  and  Low  Readiness 

■  Sample  Size  (High  and  Low  Density  Table  of  Authorized  Material  Control 
Number  [TAMCN]) 

■  Old  System 

■  New  System 

■  Recent  Mission  Change 

■  Configuration  Change 

C.  TLCM-AT  FUNCTIONAL  OVERVIEW 

TLCM-AT  is  a  technology  developed  to  assist  weapon  systems  fleet  managers 
with  evaluating,  quantifying  and  reducing  life-cycle  costs  without  adversely  impacting 
fleet  readiness  and  availability.3  This  model  is  a  holistic,  continuous-loop  representation 
of  the  life  cycle  of  any  weapons  system:  it  combines  operations,  maintenance,  and 
logistics,  as  shown  in  Figure  1.  The  inner  loop  contains  examples  of  the  inputs  to 
TLCM-AT,  and  the  items  on  the  outside  of  the  loop  represent  examples  of  outputs 
obtained  after  simulation  runs. 


3  Clockwork  Solutions,  Inc.,  “Predictive  Modeling  Technology  and  Consulting  Services  in  Support  of 
Phase  III  TLCSM,”  Final  Project  Report  (1992-2005),  submitted  to  Headquarters,  Marine  Corps 
Installations  and  Logistics,  31  December  2007,  p.  1. 
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Figure  1.  TLCM-AT  continuous-loop  model  (After)4 

An  individual  weapons  system  TLCM-AT  model  is  composed  of  six 
interactive  components5: 

■  Initialization:  Readiness  condition  and  location  of  systems  and  parts  at 
simulation  start. 

■  System  Model:  Work  breakdown  structure  of  the  system  and  its  variants. 

■  Operations  Model:  Current  and  future  operations  according  to  base 
location,  platfonn  type,  or  serialized  system. 

■  Maintenance  Model:  Actions  on  a  component  in  maintenance  as 
described  by  capacity  constraints. 

■  Sustainment  Model:  Spares,  lateral  resupply,  depot  upgrades,  and 
induction  programs. 

■  Cost  Model:  Cost  of  purchases  and  activities,  including  maintenance, 
training,  initial  and  subsequent  provisioning  of  parts,  storage,  shipping, 
and  upgrades. 


4  Clockwork  Solutions,  Inc.,  “TLCM-AT  Training  Material,”  Chapter  2,  Naaman  Gurvitz,  Ph.D. 

7  August  2007,  p.  2. 

5  Clockwork  Solutions,  Inc.,  Technical  Reference  Manual,  “Aircraft  Total  Life  Cycle  Assessment 
Software  Tool  (ATLAST),”  Version  5.O.,  (1992-2005),  Austin,  Texas,  p.  14. 
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The  system  model  consists  of  input  data  that  represents  the  structure,  location,  age 
and  status  of  components  (installed,  spares,  serviceable/unserviceable).  The  System 
Operational  Effectiveness  (SOE)  parts  usage  database  is  used  to  build  the  system 
structure  for  the  AAV,  LAV,  and  MTVR  models.  SOE  is  a  tool  developed  by 
Marine  Corps  Systems  Command  to  monitor  and  measure  system  operational 
effectiveness  attributes  on  a  wide  range  of  Marine  Corps  ground  weapons  systems.  It 
automatically  calculates  and  summarizes  key  reliability,  maintainability  and  availability 
metrics.  SOE  provided  part  usage  data  as  well  as  failure  rates.  The  rest  of  the  models  are 
built  from  a  variety  of  sources:  spreadsheets  from  Marine  Corps  Logistics  Command 
(LOGCOM),  the  Capability  Assessment  Support  Center  (CASC),  and  queries  to  the 
program  manager’s  office  for  the  individual  weapon  systems.  The  LW-155  model  does 
not  use  SOE  at  all;  it  starts  with  original  equipment  manufacturer  (OEM)  structure  and 
reliability  information.  Lastly,  the  JLTV  model  is  built  by  extracting  relevant  subsystems 
from  a  high-fidelity  LAV  model  with  an  indentured  system  structure.  Failure  rates  for 
the  JLTV  components  are  exponentially  distributed  and  in  accordance  with  historical 
LAV  maintenance  records.  For  an  exhaustive  list  and  explanation  of  TLCM-AT’s  inputs 
and  outputs,  refer  to  the  technical  manual,  which  can  be  provided  by 
Clockwork  Solutions. 

The  operations  module  consists  of  the  base  structure  and  placement  of  systems 
and  the  current  and  future  usage  rates.  Figure  2  shows  a  mapping  of  the  base  structure 
and  system  placement  for  the  LAV  variants.  The  MTVR  model  has  a  total  of  7,597 
vehicles  distributed  between  I-IV  Marine  Expeditionary  Force  (MEF),  7  MEF,  Maritime 
Prepositioning  Ships  (MPS)  and  Depot  inventory.  The  JLTV  model  uses  eight  notional 
bases  and  will  gradually  field  900  systems  among  them  from  2009-2012.  The  LW-155 
model  places  the  Marine  artillery  regiments  at  the  Operational  Level  (O-level),  in 
addition  to  the  Enhanced  Equipment  Allowance  Pool  (EEAP),  the  Anny’s  field  artillery 
school  at  Fort  Sill,  and  the  Aberdeen  Proving  Grounds.  Guns  are  fielded  according  to  the 
schedule  in  the  User’s  Logistics  Support  Summary  (ULSS). 
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Figure  2.  Base  structure  and  system  placement  in  the  LAV  model 


Operational  tempos  for  baseline  models  are  set  to  USMC  fleet  averages. 
Deployed  units  (7  MEF)  have  roughly  five  times  the  usage  rate  as  nondeployed  units. 
Units  in  stores  or  MPS  are  not  operated.  Op-tempo  for  the  LW-155  is  assumed  to  be 
1,000  km  driven  and  100  rounds  fired  per  year  for  all  each. 

The  maintenance  component  determines  the  task  times  and  effects  on  the  system 
and  its  subsystems  upon  entering  maintenance  actions.  TLCM-AT  uses  part  numbers 
(if  applicable)  to  model  preventive  maintenance,  partial  repairs,  depot  upgrades,  principle 
end-item  (PEI)  depot  rotations,  mandatory  life  limited  removals,  and 
unscheduled  failures. 

For  the  SOE-based  AAV,  LAV,  and  MTVR  models,  task  times  are  taken  from 
maintenance  allocation  charts  (MAC)  which  are  estimated  by  maintenance  personnel. 
Task  times  are  approximated  as  follows: 
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■  Tear  Times:  Fifteen  minutes  for  consumable  items;  150  minutes  for 
repairable  items. 

■  Default  Repair  Times:  Fifty  days  for  depot  level;  3.65  days  for 

intennediate  and  operational  level. 

■  Other  Repair  Times:  The  repair  cycle  time  (RCT)  from  the 

Master  Work  Schedule  (MWS)  where  applicable. 

■  Build  Times:  Only  apply  to  AAV  model  where  a  one-day  system  build 
time  is  used  to  represent  the  limited  technical  inspection  (LTI)  process  at 
the  conclusion  of  maintenance. 

■  Inspection  Times:  Constant;  the  AAV  model  has  a  one-day  system 
inspection  time  to  represent  problem  diagnosis. 

The  maintenance  task  times  in  the  LW-155  and  JLTV  models  are  of  much  higher 
fidelity.  Inspection,  tear,  repair,  and  build  times  are  set  according  to  part  or  subsystem 
using  both  measured  and  estimated  crew  times  (from  LAV  subsystems  in  the  case  of 
the  JLTV). 

Depot  capacity  constraints  in  the  AAV,  LAV,  and  MTVR  models  are  detennined 
according  to  the  actual  depot  MWS  for  Fiscal  Years  (FYs)  2007-2009.  For 
FYs  2010-2011,  the  2007-2008  schedules  are  repeated.  These  are  adjusted  according  to 
fleet  needs.  Deployed  units  have  no  depot  capacity  constraints;  all  necessary  work  is 
assumed  to  be  completed.  I-Level  capacity  constraints  are  set  proportionally  to  the 
number  of  mechanics  in  the  maintenance  battalion  table  of  organization  (T/O):  80%  for 
nondeployed  units  and  100%  for  deployed  units.  In  the  LW-155  and  JLTV  models,  there 
are  currently  no  capacity  constraints. 

In  the  sustainment  component,  a  “buy  upon  condemnation”  feature  and  USMC 
spare  inventories  can  be  controlled  from  the  TLCM-AT  analysis  control  panel.  USMC 
spare  inventories  are  apportioned  in  the  AAV,  MTVR,  and  LAV  models  as  scenarios. 
The  baseline  models,  instead,  include  an  optimized  spares  package.  The  LW-155  and 
JLTV  models  also  use  optimized  spares  packages.  None  of  the  models  currently  feature 
any  depot  upgrade  programs  or  PEI  rotations,  although  the  tool  provides  this  capability. 

Shipping-time  distributions  for  the  AAV,  MTVR,  and  LAV  models  are  fit  to 
historical  data.  Many  fits  are  lognormal.  The  LW-155  and  JLTV  models  use  appropriate 
constant  times  for  shipping  between  maintenance  levels. 
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In  the  cost  component,  TLCM-AT  enables  computation  of  the  following  total 
life-cycle  costs: 

■  Initial  provisioning 

■  Spares  storage 

■  Spares  replenishment 

■  Shipping 

■  Supply  administration 

■  Manpower 

■  Training 

■  Test  equipment  maintenance 

■  Test  equipment  space 

■  Development 

Costs  may  be  displayed  as  a  function  of  time,  as  totals,  and  per  operating  hour. 
Furthermore,  costs  related  to  individual  Line  Replaceable  Units  (LRUs)  may  be  analyzed. 
TLCM-AT  features  include: 

■  Uses  Marine  Corps’  automated  SOE  Decision  Support  Tool. 

■  Database  structure  provides  for  plug-ins  and  simulation  of  numerous 
systems  and  platforms. 

■  Provides  “what  if’  simulation  scenario  management  for  analysis  of 
life-cycle  sustainment  forecasts. 

■  Includes  operation  and  maintenance  models. 

■  Forecasts  by  part  numbers  or  serial  numbers  by  equipment  group. 

■  Maintenance  modeling-phased  inductions  include  three  options: 
scheduled  or  preventative,  causal,  or  opportunistic. 

■  Life-cycle  impact  assessments  include  three  aging  options:  aging  by 
assembly,  location,  and  repair  interval. 

■  Ability  to  initialize  system  state,  component  state,  and  life-cycle 
metric  prediction. 

■  Operational  variations  by  location. 

■  Part  interchangeability  and  substitutability  rule  manipulation. 

■  Repair  capacity  evaluation,  which  allows  for  measuring  total  repair  cycle 
time  for  individual  LRUs. 

■  Cycle  time  evaluation  (repair,  transportation,  and  order  lead  time). 
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■  Up  to  three  maintenance  levels. 

■  Time-dependent  forecasting,  which  allows  for  any  model  outputs  to  be 
displayed  in  time -based  format. 

With  the  TLCM-AT  tool,  users  may  quantify  life-cycle  costs  and  impacts  due  to 
management  decisions  regarding  equipment  configuration  changes,  remaining-life  rules 
modifications,  alternate  sparing  strategies,  adjusted  operating  hours  programs,  modified 
repair  concept,  and  age  and  reliability  factors.  Applications  for  use  include: 

■  Maintenance  concept  modifications. 

■  Deployment  strategy  assessments  (operations). 

■  Systems  degraders ’  assessment  (sensitivity). 

■  Reliability  modifications  impacts  (build  and  configure). 

■  Supply  chain  alternatives  (supply,  stock,  and  resupply). 

■  PEI  rotations  (fleet  management). 

Military  services  use  a  complex  system  of  maintenance  and  supply  to  manage 
reparable  items  in  their  inventories  using  two  key  concepts:  level  of  repair 
and  indentures.6 

Levels  of  repair  are  differentiated  by  their  varying  the  number  of  resources  and 
capabilities.  Organizational  (or  operational)  repairs  are  closest  to  the  weapons  system 
and  consist  mainly  of  diagnostics  and  remove-and-replace  maintenance  of  the  weapons 
system  itself.  Depot-level  maintenance  exists  farthest  from  the  weapons  system. 
Depot-level  maintenance  includes  diagnostics,  overhaul,  and  remanufacturing.  One  or 
more  levels  of  intermediate  repair  may  exist  between  organizational  and  depot-level 
maintenance.  TLCM-AT  models  three  levels  of  repair:  operational,  intermediate 
and  depot. 

Reparable  items  consist  of  smaller  subcomponents,  forming  an  indenture 
relationship  between  parent  parts  and  child  parts.  Having  an  indenture-level 
organizational  structure  means  an  item  can  be  repaired  at  some  intermediate  level  by 
removing  and  repairing  components  and  subcomponents.  The  indenture  relationship 

6  Bradley  E.  Anderson,.  Marvin  A..Arostegui,  and  David  L.  Lyle,  “Methods  for  Conducting  Military 
Operations  Analysis,”  Military  Operations  Research  Society,  LMI  Research  Institute,  McLean,  Virginia, 
2007,  p.352. 
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requires  logisticians  and  supply  personnel  to  balance  stock  inventories  to  stock  the  right 
expensive  items  and  smaller,  cheaper  subcomponents.  These  items  must  also  be  placed 
at  the  right  level  of  repair.  The  TLCM-AT  model  uses  the  indenture-level  concept. 
Figure  3  shows  the  basic  structure  of  the  indenture  levels  modeled  in  TLCM-AT. 


Indenture  Levels 


Platforms  Engines /LRU 


LAV 


Engine 


Systems  are 
composed  of  physical 
components  organized 
in  hierarchical 
structures 


Modules 
Module  1 


Parts 

Part  1  of 
Module  1 


Sub-Parts 
Sub -Part  1  of 
Part  1 


Sub  -Part  2  of 


Part  1 


Part  2  of 
Module  1 


Sub  -Part  of  Part 
2 


Indenture  Levels 
1 st  level  =  Platform 

2  nd  level  =  Engines 

3  rd  level  =  Modules  or  Shop 
Repairable  units 

4  th  level  =  Parts 

5  th  level  =  Subparts 


Module  N 
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Figure  3.  TLCM-AT  indenture  structure 


The  TLCM-AT  model’s  structure  and  organization  enables  the  Marine  Corps  to 
model  the  myriad  of  industry-accepted  elements  that  directly  affect  the  Ao  of  a  system, 
and  easily  run  “what  if’  scenarios  by  manipulating  data  inputs  to  examine  the  long-term 
effects  to  life-cycle  elements.  TLCM-AT  provides  a  common  tool  across  the 
Marine  Corps  to  meet  enterprise-level  concerns  and  identify  where  improvements  can  be 
made  with  regard  to  process  and  policy,  asset  management  and  performance-based 
logistics  (PBL)  validation. 
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III.  SCENARIO  DEVELOPMENT  AND  DATA  GENERATION 


A.  INTRODUCTION 

This  chapter  explains  the  scenario  and  the  data  generation  for  use  in  the  post 
experiment  analysis.  The  concepts  of  DOE  and  NOLHs  aid  in  the  analysis  of  a  complex 
model  like  TLCM-AT.  Data  is  generated  and  automated  to  support  the  analysis.  A 
fictional  LAV  scenario  is  used  as  the  basis  for  the  simulation. 

B.  DESIGN  OF  EXPERIMENTS 

TLCM  includes  everything  from  the  design,  engineering,  and  manufacturing,  to 
the  disassembly  and  disposal  of  a  system.  Life-cycle  simulation  tools  are  complex  and 
contain  many  factors  or  variables.  To  identify  robust  mixes  of  these  factors  and  their 
synergies,  over  many  scenarios,  experimental  designs  are  needed.  The  DOE  concept 
allows  researchers  to  simultaneously  vary  the  levels  of  factors  (or  inputs),  resulting  in  an 
estimate  of  factor  effects  on  response  variables  (outputs). 

To  understand  the  contributions  of  individual  factors,  each  factor  is  varied  to 
produce  all  possible  combinations.  This  is  sometimes  referred  to  as  a  full-factorial 
design,  which  considers  each  possible  factor  combination.  The  number  of  simulation 

runs  required  in  a  full-factorial  design  increases  exponentially  with  the  number  of  factors. 

12 

Lor  example,  a  full-factorial  design  of  12  factors,  each  having  ten  levels,  requires  10 
(one  trillion)  runs.  Running  this  many  combinations  is  impractical.  To  avoid  this 
difficulty,  a  base  case  scenario  may  be  compared  to  others  by  changing  one  factor  at  a 
time.  While  this  is  a  time-saving  approach,  estimates  of  interactions  (synergies)  among 
the  different  factors  may  be  overlooked,  and  conclusions  may  be  inaccurate.  Lortunately, 
these  difficulties  can  be  mitigated  by  applying  the  NOLH. 

NOLHs  allow  exploitation  of  a  large  portion  of  the  factor  space  without  requiring 
an  unrealistic  number  of  runs.  These  designs  are  nearly  orthogonal  because  a  small 
degree  of  nonorthogonality  is  allowed  in  order  to  achieve  better  space-filling.  A  design 
matrix  is  referred  to  as  nearly  orthogonal  if  the  maximum  absolute  pair-wise  correlation 
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between  any  two  columns  is  less  than  .05.  The  small  sample  of  the  NOLH  is  seen  in 
Table  1,  and  the  full  NOLH  can  be  found  in  the  Appendix. 


low  level 

1 

1 

1 

0.5 

0 

high  level 

30 

30 

30 

1.5 

10 

decimals 

0 

0 

0 

4 

4 

factor  name 

Spares 

IQ 

1  Cap 

Deg 

ST 

8 

14 

12 

0.9531 

3.3594 

27 

10 

13 

0.9609 

0.9375 

14 

23 

1 

0.7734 

4.1406 

21 

27 

10 

0.8672 

4.375 

1 

12 

17 

0.7344 

1.0156 

21 

13 

21 

0.5078 

3.9844 

12 

30 

23 

0.7891 

1 .5625 

18 

21 

27 

0.5625 

3.4375 

2 

2 

8 

0.7031 

1.9531 

30 

3 

7 

0.7656 

1.1719 

2 

29 

15 

0.7813 

3.125 

28 

30 

8 

0.8828 

1 .4063 

15 

8 

29 

0.6797 

4.0625 

23 

7 

28 

0.8516 

0.8594 

8 

16 

29 

0.7266 

4.6875 

24 

23 

30 

0.5313 

1.7188 

Table  1 .  Spreadsheet  containing  a  small  portion  of  the  NOLH  for  this  thesis 


Each  row  in  the  yellow-shaded  section  in  Table  1  represents  one  design  point  or 
simulation  run  (Spares  =  number  of  spares,  IQ  =  induction  quantity,  I  Cap  =  I-level 
capacity,  Deg  =  degradation  time,  ST  =  service  time).  The  number  under  each  column 
represents  the  level  of  that  factor  during  that  particular  design  point’s  simulation  run. 
This  analysis  required  129  rows  (design  points)  in  the  yellow-shaded  area.  After  all 
design  points  are  replicated,  the  effects  of  each  factor  and  factor  combination  can  be 
analyzed.  Figure  4  shows  all  two-way  input  combinations.  Each  cell  shows  all  input 
combinations  between  the  factors  on  the  x-axis  and  the  factors  on  the  y-axis. 
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Figure  4.  All  two-way  input  combinations 


C.  DATA  GENERATION  AND  FLOW 

This  section  briefly  describes  the  data  generation.  TLCM-AT  uses  a  Microsoft 
Access  database  for  both  inputs  and  outputs  (both  are  contained  in  the  same  file). 
Varying  levels  of  multiple  factors  over  many  design  points  can  be  tedious,  time- 
consuming,  and  virtually  impossible  for  large-scale  changes.  To  alleviate  this  problem,  a 
procedure  is  developed  to  efficiently  change  the  levels  of  multiple  factors  in  the  database, 
send  them  to  the  TLCM-AT  for  a  simulation  run,  and  gather  the  results.  This  procedure 
uses  a  Java  program.  A  representation  of  the  data  generation  is  provided  in  Figure  5. 
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Figure  5.  Diagram  of  data  generation  process 


The  Java  program  receives  the  NOLH  design  matrix  and  the  baseline  database  for 
each  scenario.  The  program  reads  the  first  design  point  row  and  the  levels  for  each  factor 
and  then  goes  into  the  database  to  adjust  each  factor  in  the  appropriate  input  table.  Next, 
the  updated  database  is  sent  to  TLCM-AT  for  a  simulation  run  on  that  single  design 
point.  The  output  data  generated  from  the  simulation  run  is  then  collected  by  the 
program  and  written  to  a  comma-separated-file.  This  procedure  is  repeated  automatically 
for  each  design  point.  Upon  completion  of  all  simulation  runs,  a  single  file  containing 
the  data  of  interest  for  each  design  point  is  available  for  post-analysis.  Detailed  code  for 
this  process  can  be  found  in  an  NPS  thesis  scheduled  for  completion  in  September  2008, 
by  Lieutenant  Commander  Alberto  A.  Garcia,  United  States  Navy. 

D.  SCENARIO  DEVELOPMENT 

This  section  explains  the  scenarios  used  for  this  thesis.  These  scenarios  represent 
possible  COAs  for  a  notional  contingency  deployment  for  the  LAV.  Each  scenario  is 
modeled  separately  in  four  different  databases  to  represent  the  particular  plan  for  the 
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replacement  of  two  trouble  parts.  The  scenarios  were  modeled  by  a  modeler  and  analyst 
from  Clockwork  Solutions  (Dr.  Peter  Figliozzi). 

1.  Notional  Background 

The  Marine  Corps  is  given  a  warning  order  to  rapidly  deploy  for  combat 
operations  in  a  tropical  region.  The  mission  details  and  terrain  drive  the  Marine  Corps  to 
include  at  least  a  battalion  of  LAVs  in  the  force  mix.  The  deployment  consists  of  a  lead 
LAV  battalion  and  a  follow-on  LAV  battalion. 

From  historical  data  of  combat  operations  in  hot,  humid  environments,  LAV 
maintainers  anticipate  numerous  unscheduled  removals  rates  of  two  computerized  LRUs 
on  the  LAV  weapons  system: 

■  OT  70227500 1 ,  SENSOR  UNIT,  LASER  DESIGNATOR 

■  OT  70226 1001,  CONTROL  DISPLAY  UNIT 

The  problem  with  the  computerized  LRUs  first  becomes  apparent  only  a  few 
weeks  after  the  lead  battalion’s  arrival  in  theater.  This  maintenance  problem  is  a 
detriment  to  the  Marines’  ability  to  accomplish  their  mission.  Given  time  to  assess  the 
situation,  three  COAs  are  suggested: 

COA  1:  Send  a  large  number  of  spares  with  the  follow-on  battalion. 

COA  2:  Invest  in  improved  LRUs.  The  research  and  development  (R&D) 

cost  for  the  one -month  R&D  program  is  $1M,  and  the  new  LRUs 
cost  an  additional  1.5  times  the  cost  of  the  old  type.  Bring  a  lesser 
number  of  the  improved  spares  and  install  them  whenever  the  old 
type  is  removed.  When  the  old  type  is  condemned,  buy  the 
new  type. 

COA  3:  A  variant  of  COA  2,  except  that  the  old  type  is  purchased 

whenever  the  LRUs  are  condemned.  No  money  is  invested  in  the 
new  LRUs.  The  idea  is  to  save  some  money,  while  maintaining 
the  same  level  of  Ao  and  maintenance  utilization. 

2.  Simulation  Scenarios 

This  subsection  provides  some  technical  details  of  each  of  how  the  scenarios 
(or  COAs)  are  modeled  in  the  TLCM-AT  database.  Four  scenarios  are  used.  The  first  is 
a  baseline  scenario  with  no  corrective  action  taken  (maintaining  the  status  quo). 
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Scenarios  1-3  will  be  compared  against  the  baseline  scenario.  A  Stock  Record  Account 
Number  (SRAN)  is  used  to  indicate  the  location  of  the  LAV.  For  example,  SRAN  10001 
indicates  that  the  LAV  is  located  with  I-MEF,  and  SRAN  10007  indicates  that  the  LAV  is 
located  with  a  deployed  MEF.  Each  scenario  is  detailed  below: 

LAV  3042  v5  Tropical  Baseline  Scenario: 

This  is  the  baseline  scenario  that  maintains  the  status  quo: 

■  Baseline  model  is  LAV  3042  v5  (SOE  based,  LRU-only  model). 

■  SRAN  10007  LA  Vs  are  turned  off  (zero  optempo)  except  during  the 
following  times:  2008  Q4  -  2010  Q1  inclusive  (the  contingency  period). 

■  The  optempo  for  all  SRAN  10007  LA  Vs  during  the  contingency  period  is 
2,000  hours/year. 

■  An  initial  force  of  88  LAVs  (including  45  LAV-25s)  at  SRAN  10007 
operates  throughout  the  contingency  period. 

■  All  of  SRAN  10001  LAVs  (111  total  LAVs  including  51  LAV-25s) 
redeploy  to  SRAN  10007  in  2009  Ql.  In  2010  Ql,  they  redeploy  back  to 
SRAN  10001.  Thus,  they  are  active  in  SRAN  10007,  together  with  the 
initial  force,  from  2009  Ql  -  Q4  inclusive. 

■  The  failure  rate  for  OT  702275001,  SENSOR  UNIT,  LASER 
DESIGNATOR,  is  set  to  4.3  per  10,000  operating  hours,  only  at 
SRAN  10007.  Elsewhere,  it  is  0.43. 

■  The  failure  rate  for  OT  70226 1001,  CONTROL  DISPLAY  UNIT,  is  set  to 
4.2  per  10,000  operating  hours,  only  at  SRAN  10007.  Elsewhere,  it 
is  0.42. 

LAV  3042  v5  Tropical  -  Scenario  1: 

This  scenario  represents  the  plan  to  bring  old-type  spares  with  the 
follow-on  battalion: 

■  Add  numerous  amounts  of  spares  to  the  deployment. 

■  Quantity  of  100  of  the  two  troubling  LRUs  are  added  to  SRAN  10007  as 
spares,  during  2009  Ql. 

LAV  3042  v5  Tropical  -  Scenario  2: 

This  scenario  includes  investment  in  improved  SENSOR  UNIT  and  CONTROL 
DISPLAY  UNIT;  however,  limited  quantities  are  immediately  available  (50  of  each  for 
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the  deployment).  They  are  taken  on  deployment  as  spares.  Additionally,  when  the  old 
types  are  condemned,  the  new  type  is  bought  as  replacements. 


■  Added  OT  7022275002  and  702261002  to  ^Object  type  table  (duplicates 
of  00 1  versions,  added  a  “2”  to  the  end  of  the  part  number  to  differentiate 
it  from  the  old  part). 

■  Changed  *Preferred  buys  table  for  001  type  to  002  type. 

■  Added  002  type  to  *Preferred  buys  table. 

■  Duplicated  *Server  times  table  for  002  types  (made  same  as  001  types). 

■  Changed  *Unscheduled  removal  rates  table  to  0.4  for  both  002  types 
(all  locations). 

■  Changed  *Slots  by  type  table  for  LAV-25  to  prefer  002  variant. 

■  Made  price  for  type  002  in  *LCC  costs  table  1.5  times  the  001  price. 

■  Quantity  of  50  of  the  new  type  (002)  of  the  two  trouble  LRUs  added  to 
SRAN  10007  as  spares,  2009  Ql. 

LAV  3042  v5  Tropical  -  Scenario  3: 

This  scenario  is  the  same  as  COA  2,  except  the  initial  purchase  of  50  new  002 
types  occurs  only  once.  No  new  types  are  ever  purchased  upon  condemnation  (only  the 
001  types  are  purchased,  whenever  a  001  or  002  type  is  condemned). 

■  Copied  COA  2  database  to  create  COA  3  database. 

■  Changed  *Preferred  buys  table  for  001  type  back  to  001  type. 

■  Changed  *Preferred  buys  table  for  002  type  to  001  type. 

E.  SIMULATION  RUNS 


Five  factors  are  varied  during  each  scenario.  The  levels  of  each  factor  are 
summarized  in  Table  2: 


FACTOR 

HIGH 

RANGE 

LOW 

RANGE 

DESCRIPTION 

Spares 

1 

30 

Spares  level  set  to  the  value  in  the  NOLH  column 

Induction  Quantity 

1 

30 

Induction  Quantity  set  to  the  value  in  the  NOLH  column 

I-Level  Capacity 

1 

30 

Capacity  at  the  I-Level  set  to  the  value  in  the  NOLH  column 

Degradation  Rate 

0.5 

1.5 

Unscheduled  Removal  Rate  multiplied  by  the  value  in  the  NOLH  column 

Service  Time 

0 

10 

Service  Time  (time  to  repair  part)  multiplied  by  the  value  in  the  NOLH  column 

Table  2.  Range  of  changes  for  the  factors  studied. 
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The  level  of  the  five  factors  is  adjusted  for  25  LRUs,  which  are  chosen  as  the  top 
25  degraders.  They  are  the  25  parts  that  cause  the  most  problems  during  the  life-cycle  of 
the  LAV-25  in  the  baseline  model.  The  top  degraders  are  ranked  using  a  formula 
provided  in  the  final  project  report  issued  by  Clockwork  Solutions  on  3 1  December  2007: 

Score  =  (Waiting  time)  *  (Requests)  *  (Unavailability  +1) 

Waiting  time  is  the  average  logistics  response  time.  Requests  is  the  number  of  times  the 
part  is  requested  by  the  fleet.  Unavailability  is  the  fraction  of  part-needed-but-not-spared 
occurrences  by  location.  Unity  (+1)  is  added  to  Unavailability  so  that  parts  with  frequent 
failures  are  appropriately  spared. 

The  primary  measure  of  effectiveness  (MOE)  is  Ao.  After  each  simulation  run, 
Ao  is  gathered  for  each  quarter,  for  a  period  of  five  years  from  the  *out  Availability 
output  table  from  the  TLCM-AT.  (It  should  be  noted  that  the  515 out  Availability  table 
gives  the  Ao  as  the  mean  of  the  number  of  replications.) 

Each  of  the  four  scenarios  (the  baseline  plus  the  three  COAs)  is  simulated  in 
TLCM-AT  using  the  DOE  concept  and  the  NOLH,  as  previously  discussed.  For  each 
scenario,  the  129  design  points  are  formed  from  the  same  NOLH  design  matrix.  For  each 
design  point,  30  replications  are  completed.  The  constraint  on  the  number  of  replications 
is  run  time.  Each  design  point  takes  approximately  30  minutes  to  run.  In  summary: 

■  Total  design  points:  4  scenarios  with  129  designs  point  each  =  516 

■  Total  replications:  129  design  points  with  30  replications  =  15,480 

■  Total  time  for  all  simulation  runs:  516  design  points  for  30  minutes  each 
=  258  hours  (10+  days). 

■  Simulation  runs  executed  on  two  standard  desktop  processors  for  a  total 
duration  of  approximately  five  days. 
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IV.  ANALYSIS 


A.  INTRODUCTION 

This  chapter  describes  the  data  collection  and  post  processing.  In  accordance 
with  the  scenarios  described  in  Chapter  III,  the  analysis  centers  on  evaluation  of  Ao  at  the 
end  of  quarters  12  and  20:  quarter  12  is  the  end  of  the  forecasted  end  of  the  contingency, 
and  quarter  20  is  the  end  of  the  five-year  period  of  evaluation  (five  quarters  of 
high-tempo  operation  and  normal  operation  for  the  remaining  time).  Section  B  contains 
an  interpretation  of  the  results  produced  by  TLCM-AT  after  running  each  scenario,  and  a 
discussion  of  the  conclusions  that  may  be  drawn  from  the  results.  Section  C  gives 
detailed  analysis  and  discussion  of  the  results  produced  after  simulation  runs  with  DOE 
implementation.  Throughout  the  analysis,  insight  into  the  TLCM  process  is  more 
important  than  specific  numerical  results.  The  focus  of  this  chapter  is  to  show  how 
TLCM-AT,  combined  with  DOE,  can  be  used  by  analysts  to  maximize  the  insight  drawn 
from  discrete  event  simulation. 

B.  TLCM-AT  RESULTS 

The  analysis  begins  by  examining  TLCM-AT  as  a  stand-alone  tool,  without 
implementation  of  DOE.  Each  scenario  run  is  simulated  through  TLCM-AT  using 
100  replications.  At  the  completion  of  the  four  simulation  runs  (one  for  each  COA  and 
one  for  the  baseline),  post-processing  applied  with  TLCM-AT ’s  graphical-user-interface 
(GUI)  and  its  built-in  analytical  utilities.  The  TLCM-AT  graphs  are  best  viewed  in  color. 

In  an  example  of  quick-turn  analysis  using  TLCM-AT,  a  cost  comparison  is  first 
completed.  Results  from  each  COA,  as  they  relate  to  cost,  are  depicted  in  Figures  6 
and  7. 
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Time  Dependent  O&S  Cost  Graph 


COA1 

COA2 

COA3 


Date 


Figure  6.  Operation  and  Sustainment  (O&S)  cost  graph 
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Oveiall  Total  O&S  Cost  Graph 


□  COA1 

□  COA2 

□  COA  3 


Figure  7.  Overall  total  O&S  graph 


Figure  6  represents  time-dependent  O&S  costs  for  spares  replenishment  for  each 
COA  over  the  five-year  period.  The  peak  evident  in  the  graph  indicates  the  time  period 
in  which  the  contingency  operations  are  being  conducted.  Figure  7  depicts  the  overall 
total  O&S  cost  for  each  COA.  COA  2  saves  $3.2  million  compared  to  COA  3  and 
$15.8  million  compared  to  COA  1.  The  results  quantify  the  immediate  value  of  the 
engineering  change  program  (ECP)  to  improve  the  two  trouble  parts.  Even  though  the 
price  of  improved  parts  is  50%  greater  than  the  price  of  the  old  parts,  the  ECP  immunizes 
the  LAV -25  fleet  against  humidity  problems  in  the  future  and  saves  money  in 
the  meantime. 
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Next,  a  comparison  is  done  to  reflect  how  each  COA  affects  Ao.  This  comparison 
includes  the  baseline  scenario,  representing  the  decision  to  maintain  the  status  quo. 
Figure  8  depicts  time-dependent  Ao  of  the  LAV-25  for  each  COA. 

Time  Dependent  Availability  l>y  System  Type(s)j 

-  System  Type  E0947  with  Base  Base  7  -  Jungle  from  Baseline(Status  quo) 

-  System  Type  E0947  with  Base  Base  7  -  Jungle  from  COA1 

-  System  Type  E0947  with  Base  Base  7  -  Jungle  from  COA2 

-  System  Type  E0947  with  Base  Base  7  -  Jungle  from  COA  3 


Date 


Figure  8.  Time-dependent  Ao 

As  Figure  7  reveals,  each  COA  (1-3)  maintains  relatively  the  same  Ao  throughout 
the  contingency  timeframe,  as  well  as  throughout  the  full  20-quarter  period.  Also, 
maintaining  the  status  quo  will  greatly  reduce  Ao  during  the  contingency,  as  depicted 
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with  the  red  (lowest)  line  in  Figure  7.  In  regard  to  Ao,  it  is  evident  that  taking  some 
action,  whether  it  is  COA  1,  2,  or  3,  is  much  more  preferable  to  maintaining  the 
status  quo. 

Adding  more  old  spares  or  introducing  new  spares  into  the  maintenance  system 
will  undoubtedly  have  an  effect  on  a  repair  facility.  TLCM-AT  can  display  this  effect  by 
examining  the  number  of  maintenance  tasks  performed  during  the  contingency  on  the 
laser  designator  unit.  COA  2  clearly  produces  the  lightest  workload  for  the  maintenance 
facility,  as  depicted  by  the  blue  (lowest)  line  in  Figure  9.  Base  Shop  7  in  Figure  9 
represents  the  repair  facility  designated  for  maintenance  on  the  LAV -25s  during 
the  contingency. 

Time  Dependent  Tasks  Peifonnedj 

-  Repair  for  WUC  LASER  DESIG  SENSOR  UNIT  with  Base  Shop  7  -  Jungle  Int  from  COA1 

-  Repair  for  WUC  LASER  DESIG  SENSOR  UNIT  with  Base  Shop  7  -  Jungle  Int  from  COA2 

1  Repair  for  WUC  LASER  DESIG  SENSOR  UNIT  with  Base  Shop  7  -  Jungle  Int  from  COA  3 


Date 


Figure  9.  Number  of  maintenance  tasks  performed  on  the  laser  designator  unit 
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From  the  analysis,  it  is  evident  that  COA  2  is  the  preferred  COA.  It  has  lower 
overall  O&S  costs  for  spares  replenishment,  and  provides  the  least  amount  of  impact  on 
the  repair  facility.  These  effects  are  accomplished  with  no  negative  impact  on  Ao. 

The  preceding  analysis  and  discussion  is  an  example  of  the  quick-tum  type  of 
analysis  that  can  be  accomplished  with  TLCM-AT.  However,  it  must  be  remembered 
that  each  COA  had  to  be  modeled  separately  in  the  Access  database  before  execution  of 
the  simulation.  Depending  on  the  user’s  familiarity  with  the  TLCM-AT  modeling 
process,  this  could  be  a  fairly  time-consuming  task. 

C.  TLCM-AT  WITH  DOE  IMPLEMENTATION 

This  section  discusses  the  analytical  process  and  results  after  simulation  with 
DOE  implementation  and  insights  gained.  As  in  the  previous  analysis,  the  focus  in  this 
section  is  directed  at  the  Ao  in  quarters  12  and  20.  This  section  contains  regression 
analysis,  t-Tests  for  two-sample  means  and  an  evaluation  of  the  best  design  points.  It 
should  be  noted  here  that  the  practical  change  in  Ao  after  the  simulation  runs  is  very 
minimal.  In  other  words,  changes  are  too  small  to  effect  a  decision.  This  result  is  in  line 
with  the  results  seen  in  Section  B.  The  ranges  of  Ao  are  depicted  in  Table  3. 


Quarter  12 

Quarter  20 

Baseline 

COA1 

COA2 

COA3 

Baseline 

COA1 

COA2 

COA3 

Max  Ao 

82.18% 

83.00% 

82.83% 

83.15% 

90.19% 

90.31% 

90.11% 

90.41% 

Min  Ao 

80.39% 

81.23% 

80.76% 

87.43% 

88.73% 

88.84% 

88.69% 

Table  3.  Ao  ranges  per  COA  for  quarters  12  and  20 

The  maximum  and  minimum  Ao  in  all  simulation  runs,  across  all  COAs  and 
design  points  is  depicted  in  Table  4. 


j  Quarter  12 

Quarter  20 

Max  Ao 

Min  Ao 

83.15% 

80.39% 

MaxAo  90.41% 

MinAo  87.43% 

Table  4.  Ao  ranges  across  all  COAs  and  design  points 
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1.  Regression  Analysis 

Regression  is  a  technique  used  for  the  modeling  and  analysis  of  numeric  data 
consisting  of  values  of  a  dependent  response  variable  (Ao)  and  one  or  more  independent 
explanatory  variables  (spares,  induction  quantity,  capacity,  degradation  time,  service 
time).  Ao  is  the  only  response  variable  used  because  of  time  constraints  to  complete  this 
study.  The  dependent  variable  in  the  regression  equation  is  modeled  as  a  function  of  the 
independent  variables.  Regression  analysis  shows  the  independent  variables  that  are 
statistically  significant  to  the  dependent  variable.  Among  many  independent  variables, 
only  a  few  of  them  may  be  significant  in  detennining  the  measure  of  interest. 

This  analysis  uses  stepwise  regression  to  evaluate  the  data  obtained  from  the 
simulation  runs  of  the  COAs  discussed  in  Chapter  III.  The  basic  procedures  for  stepwise 
regression  involve: 

1 .  Identifying  an  initial  model. 

2.  Iteratively  stepping,  or  repeatedly  altering,  the  model  at  the  previous  step 
by  adding  or  removing  an  independent  variable. 

3.  Terminating  the  search  when  stepping  is  no  longer  possible  given  the 
stepping  criteria  or  when  a  specified  number  of  steps  have  been  reached. 

The  forward  selection  approach  is  used  in  this  analysis.  Forward  selection 
involves  starting  with  no  variables  in  the  model  and  trying  out  the  variables  one  by  one, 
and  including  them  if  they  are  statistically  significant. 

Thirty-two  regression  models  are  evaluated.  One  model  is  developed  for 
quarter  12  and  another  for  quarter  20  for  each  scenario.  Regression  determines 
significant  factors  affecting  Ao,  and  the  variance  in  Ao,  explained  by  the  five  varied 
factors.  The  amount  of  variance  explained  by  the  factors  is  represented  by  the  coefficient 
of  determination,  or  R  ;  as  the  R"  increases,  the  amount  of  explained  variance  in  the 
statistical  model  increases.  A  low  R"  indicates  there  could  potentially  be  more  factors 
affecting  the  outcome  than  those  modeled.  The  R  from  all  stepwise  regressions  is  shown 
in  Table  5. 
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COA 

QTR 

Main  Effects 

2- way  Interaction 

3 -way  Interaction 

Quadratic  | 

Base  Scenario 

12 

Base  Scenario 

20 

0.65 

0.74 

0.79 

0.81 

Scenario  1 

12 

0.61 

0.71 

0.75 

0.76 

Scenario  1 

20 

0.04 

0.18 

0.22 

0.24 

Scenario  2 

12 

0.15 

0.26 

0.32 

0.4 

Scenario  2 

20 

0.02 

0.05 

0.2 

0.23 

Scenario  3 

12 

0.3 

0.47 

0.51 

0.62 

Scenario  3 

20 

0.02 

0.03 

0.04 

0.07 

Table  5. 


2 

R  from  all  stepwise  regressions 


2 

Table  5  shows  relatively  low  R"  values  for  scenarios  1-3,  especially  during 
quarter  20.  Furthermore,  the  R"  increases  in  all  cases  as  interaction  and  polynomial 
(quadratic)  terms  are  added  to  the  model.  The  increase  in  R  shows  that  including  only 
main  effects  does  not  capture  the  complete  picture;  interaction  terms  reveal  synergies 
between  the  modeled  factors.  Practically,  this  shows  that  adjusting  only  one  or  two 
factors  independently  may  not  be  the  best  solution  if  seeking  to  maximize  Ao. 
Furthermore,  this  form  of  regression  cannot  adequately  capture  the  richness  of  TLCM- 
AT;  other  factors  added  to  the  model  will  increase  R".  In  the  cases  where  R“  is  low,  the 
results  show  that  other  factors  should  be  considered. 

Scenario  two  quarter  12  is  used  as  an  example.  The  following  stepwise  regression 
includes  all  two-way  interaction  and  three-way  interaction  terms.  Table  6  shows  the 
summary  of  fit  and  parameter  estimates  produced  from  the  regression  are  shown  in 
Table  7. 


Summary  of  Fit 

RSquare 

0.400167 

RSquare  Adj 

0.216545 

Root  Mean  Square  Error 

0.00252 

Mean  of  Response 

0.822325 

Observations  (or  Sum  Wgts) 

129 

Table  6.  Summary  of  fit  for  scenario  2  quarter  12 
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Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

Intercept 

0.8214981 

0.001445 

568.48 

<.0001* 

Spares 

0.0001396 

2.959e-5 

4.72 

<.0001* 

IQ 

1 ,4267e-5 

2.81 1e-5 

0.51 

0.6129 

Capacity 

-0.000043 

2.723e-5 

-1.58 

0.1176 

Degradation 

-0.000271 

0.000799 

-0.34 

0.7355 

ServTime 

-0.000042 

8.633e-5 

-0.49 

0.6283 

(Spares-15.5039)*(IQ-15.5039) 

-4.132e-7 

3.01 5e-6 

-0.14 

0.8913 

(Spares-15.5039)*(Capacity-15.5039) 

-2.522e-6 

4.275e-6 

-0.59 

0.5566 

(Spares-15.5039)*(Degradation-1.00001) 

0.0002071 

0.00011 

1.88 

0.0634 

(Spares-15.5039)*(Serv  Time-5. 00001) 

0.0000195 

9.31e-6 

2.09 

0.0388* 

(IQ-15.5039)*(Capacity-15.5039) 

-4.408e-6 

3.632e-6 

-1.21 

0.2279 

( IQ- 1 5.5039)*(  Degradation- 1 .00001 ) 

-0.000269 

0.000133 

-2.03 

0.0452* 

(IQ-15. 5039)*(ServTime-5. 00001) 

4.8612e-7 

9.094e-6 

0.05 

0.9575 

(Capacity-15.5039)*(Degradation-1.00001) 

0.0000949 

8.844e-5 

1.07 

0.2859 

(Capacity-15.5039)*(ServTime-5. 00001) 

-1.661e-6 

0.000012 

-0.14 

0.8894 

(Degradation-1 .00001  )*(ServTime-5. 00001 ) 

0.0002695 

0.000295 

0.91 

0.3628 

(Spares-15.5039)*(IQ-15.5039)*(Capacity-15.5039) 

-1.221e-7 

4.666e-7 

-0.26 

0.7942 

(Spares-15.5039)*(IQ-15.5039)*(Degradation-1.00001) 

-2.661  e-5 

1.363e-5 

-1.95 

0.0537 

(Spares-1 5. 5039)*(IQ-1 5. 5039)*(Serv  Time-5. 00001) 

1.7182e-6 

1.112e-6 

1.55 

0.1255 

(Spares-15.5039)*(Capacity-15.5039)*(Degradation-1.00001) 

6.7712e-6 

1.366e-5 

0.50 

0.6211 

(Spares-1 5. 5039)*(Capacity-1 5. 5039)*(Serv  Time-5. 00001) 

-1.348e-7 

1.369e-6 

-0.10 

0.9218 

(Spares-1 5. 5039)*(Degradation-1. 00001  )*(Serv  Time-5. 00001) 

-0.000046 

3.86e-5 

-1.19 

0.2371 

(IQ-15.5039)*(Capacity-15.5039)*(Degradation-1. 00001) 

-2.051  e-6 

1 ,385e-5 

-0.15 

0.8826 

(IQ-15.5039)*(Capacity-15.5039)*(Serv  Time-5. 00001) 

-1.985e-6 

1.198e-6 

-1.66 

0.1007 

( IQ- 1 5.5039)*(  Degradation- 1 .00001  )*(ServTime-5.00001 ) 

-5.255e-5 

3.982e-5 

-1.32 

0.1900 

(Capacity-1 5. 5039)*(  Degradation-!  00001  )*(ServTime-5. 00001) 

0.0000417 

0.000039 

1.07 

0.2866 

(Spares-1 5. 5039)*(Spares-1 5. 5039) 

-1.067e-5 

4.502e-6 

-2.37 

0.0197* 

( IQ- 1 5.5039)*(  IQ- 1 5.5039) 

-5.518e-6 

4.21e-6 

-1.31 

0.1931 

(Capacity-15.5039)*(Capacity-1 5.5039) 

-8.746e-7 

4.012e-6 

-0.22 

0.8279 

(Degradation-1.00001)*(Degradation-1. 00001) 

0.0050247 

0.003508 

1.43 

0.1553 

(ServTi  me-5. 00001  )*(ServTime-5. 00001) 

4.5964e-5 

3.744e-5 

1.23 

0.2225 

Table  7.  Parameter  estimates  for  scenario  2  quarter  12 

Number  of  spares  is  the  dominant  factor  in  the  model.  Forty  percent  of  the 
variance  is  explained.  Table  6  shows  the  summary  of  fit  and  parameter  estimates 
produced  from  the  regression  are  shown  in  Table  7. 

The  significant  factors  in  determining  Ao  (as  shown  in  Table  7  with 
accompanying  asterisk)  are: 

■  Spares 

■  Spares  x  Spares 

■  Spares  x  Service  Time 

■  Induction  Quantity  X  Degradation 
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A  final  model  is  created  by  removing  all  insignificant  factors.  Induction  Quantity 
X  Degradation  is  now  an  insignificant  factor.  The  R“  drops  from  40%  to  24%  of  the 
variance  explained  by  the  model.  Number  of  spares  is  the  dominant  factor  in  the  final 
model.  Table  8  shows  the  summary  of  fit  and  parameter  estimates  produced  from  the 
regression  are  shown  in  Table  9. 


Summary  of  Fit 

RSquare 

0.242189 

RSquare  Adj 

0.224002 

Root  Mean  Square  Error 

0.002508 

Mean  of  Response 

0.822325 

Observations  (or  Sum  Wgts) 

129 

Table  8  Final  model  summary  of  fit  for  scenario  2  quarter  12 


Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

Intercept 

0.8213247 

0.000523 

1571.2 

<.0001* 

Spares 

0.0001175 

2.61 1e-5 

4.50 

<.0001* 

(Spares-1 5.5039)*(ServTime-5.00001 ) 

2.2623e-5 

8.165e-6 

2.77 

0.0064* 

(Spares-15.5039)*(Spares-15.5039) 

-1.148e-5 

3.434e-6 

-3.34 

0.0011* 

Table  9  Final  model  parameter  estimates  for  scenario  2  quarter  12 

The  significant  factors  in  determining  Ao  (as  shown  in  Table  9  with 
accompanying  asterisk)  are: 

■  Spares 

■  Spares  x  Spares 

■  Spares  x  Service  Time 

The  coefficients  are  small;  this  indicates  that  these  terms,  while  statistically 
significant  in  the  model,  have  minimal  practical  impact  on  Ao.  In  other  words,  the 
increase  in  Ao  for  each  spare  added  is  too  small  to  effect  a  decision.  Similar  conclusions 
are  drawn  from  the  other  COAs  as  well. 

2.  Two-Sample  Comparison 

Following  regression  analysis,  paired  two-sample  t-Tests  are  done  between  each 
scenario.  The  purpose  of  a  t-Test  is  to  determine  whether  or  not  two  population  means 
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are  equal.  The  two-sample  t-Test  is  performed  on  two  data  sets  that  are  assumed  to  have 
been  drawn  from  populations  that  follow  a  nonnal  distribution  with  constant  variance. 
However,  for  sufficiently  large  samples  there  are  moderate  to  robust  departures  from 
these  assumptions.  The  data  sets  used  here  are  the  Ao  values  generated  for  each  scenario 
over  the  129  design  points.  Six  t-Tests  are  completed  to  cover  all  possible  combinations: 

■  Base  -  Scenario  1 

■  Base  -  Scenario  2 

■  Base  -  Scenario  3 

■  Scenario  1  -  Scenario  2 

■  Scenario  1  -  Scenario  3 

■  Scenario  2  -  Scenario  3 

In  each  test,  the  hypothesized  mean  difference  is  zero  (no  difference  in  population 
means).  Tables  10  and  11  depict  the  results  of  all  paired  two-sample  t-Tests  for 
differences  in  Ao  after  quarters  12  and  20,  respectively. 
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t-Test:  Paired  Two  Sample  for  Means 

Quarter  12 

Base 

COA1 

Mean 

0.811409591 

0.818230432 

Variance 

1.1044E-05 

2.78253E-05 

Observations 

129 

129 

Hypothesized  Mean  Difference 

0 

df 

128 

P(T<=t)  two-tail 

4.65353E-33 

t  Critical  two-tail 

1.978670823 

COA1 

COA2 

Mean 

0.818230432 

0.822325324 

Variance 

2.78253E-05 

8.1056E-06 

Observations 

129 

129 

Hypothesized  Mean  Difference 

0 

df 

128 

P(T<=t)  two-tail 

2.64123E-15 

t  Critical  two-tail 

1.978670823 

COA2 

COA3 

Mean 

0.822325324 

0.821494436 

Variance 

8.1056E-06 

1.70177E-05 

Observations 

129 

129 

Hypothesized  Mean  Difference 

0 

df 

128 

P(T<=t)  two-tail 

0.022398628 

t  Critical  two-tail 

1.978670823 

COA1 

COA3 

Mean 

0.818230432 

0.821494436 

Variance 

2.78253E-05 

1.70177E-05 

Observations 

129 

129 

Hypothesized  Mean  Difference 

0 

df 

128 

P(T<=t)  two-tail 

8.1068E-12 

t  Critical  two-tail 

1.978670823 

Base 

COA2 

Mean 

0.811409591 

0.822325324 

Variance 

1.1044E-05 

8.1056E-06 

Observations 

129 

129 

Hypothesized  Mean  Difference 

0 

df 

128 

P(T<=t)  two-tail 

3.37578E-65 

t  Critical  two-tail 

1.978670823 

Base 

COA3 

Mean 

0.811409591 

0.821494436 

Variance 

1.1044E-05 

1.70177E-05 

Observations 

129 

129 

Hypothesized  Mean  Difference 

0 

df 

128 

P(T<=t)  two-tail 

2.87256E-54 

t  Critical  two-tail 

1.978670823 

Table  10.  t-Test  results  for  difference  in  Ao  for  12th  quarter 
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t-Test:  Paired  Two  Sample  for  Means 

Quarter  20 

Base 

COA1 

Mean 

0.89040315 

0.895018571 

Variance 

3.75235E-05 

8.24482E-06 

Observations 

129 

129 

Hypothesized  Mean  Difference 

0 

df 

128 

P(T<=t)  two-tail 

3.30143E-13 

t  Critical  two-tail 

1.978670823 

COA1 

COA2 

Mean 

0.895018571 

0.895466579 

Variance 

8.24482E-06 

6.7266E-06 

Observations 

129 

129 

Hypothesized  Mean  Difference 

0 

df 

128 

P(T<=t)  two-tail 

0.177946263 

t  Critical  two-tail 

1.978670823 

COA2 

CO  A3 

Mean 

0.895466579 

0.895127699 

Variance 

6.7266E-06 

8.15099E-06 

Observations 

129 

129 

Hypothesized  Mean  Difference 

0 

df 

128 

P(T<=t)  two-tail 

0.332484251 

t  Critical  two-tail 

1.978670823 

COA1 

COA3 

Mean 

0.895018571 

0.895127699 

Variance 

8.24482E-06 

8.15099E-06 

Observations 

129 

129 

Hypothesized  Mean  Difference 

0 

df 

128 

P(T<=t)  two-tail 

0.745610006 

t  Critical  two-tail 

1.978670823 

Base 

COA2 

Mean 

0.89040315 

0.895466579 

Variance 

3.75235E-05 

6.7266E-06 

Observations 

129 

129 

Hypothesized  Mean  Difference 

0 

df 

128 

P(T<=t)  two-tail 

4.25768E-14 

t  Critical  two-tail 

1.978670823 

Base 

CO  A3 

Mean 

0.89040315 

0.895127699 

Variance 

3.75235E-05 

8.15099E-06 

Observations 

129 

129 

Hypothesized  Mean  Difference 

0 

df 

128 

P(T<=t)  two-tail 

1.87398E-12 

t  Critical  two-tail 

1.978670823 

Table  1 1 .  t-Test  results  for  difference  in  Ao  for  20th  quarter 
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The  low  p-value  calculated  for  each  comparison  reveals,  in  all  six  cases,  a 
statistically  significant  difference  in  the  mean  Ao  between  each  scenario  for  quarter  12. 
However,  the  higher  p-values  evident  in  quarter  20,  for  comparison  between  scenarios  1, 
2,  and  3,  show  that  there  is  no  statistically  significant  difference  in  the  mean  Ao. 

The  statistical  significance  shown  in  quarter  12  does  not  necessarily  imply 
practical  significance.  Because  the  mean  Ao  in  all  cases  are  relatively  close,  the  mean 
difference  is  not  practically  significant;  going  from  81.8%  to  82.1%  Ao  is  not  significant 
enough  to  make  practical  impacts  on  the  overall  operational  picture. 


3.  Design  Point  Evaluation 

Additional  insight  can  be  obtained  by  taking  a  closer  look  at  the  design  points 
producing  the  highest  Ao.  The  specific  factor  settings  in  the  best  design  points  can  be 
used  to  guide  a  decision  maker  when  determining  the  level  to  set  for  specific  factors.  The 
design  points,  with  their  associated  factor  settings,  that  produced  the  highest  Ao  for 
scenario  two  are  depicted  in  Table  12. 


Rank 

Design  Point  # 

Spares 

IQ 

Capacity 

Degradation 

ServTime 

1 

86 

22 

Quarter  12 

24 

6 

0.5469 

5.0781 

2 

71 

10 

18 

10 

1.4922 

6.0156 

3 

95 

12 

17 

1 

0.8047 

5.1563 

4 

53 

10 

7 

18 

1.0078 

9.5313 

5 

18 

29 

11 

9 

1.2813 

4.2188 

6 

123 

11 

18 

27 

0.6484 

4.2969 

7 

94 

22 

17 

8 

0.9453 

6.4063 

8 

13 

15 

8 

29 

0.6797 

4.0625 

9 

51 

13 

16 

10 

1.2891 

5.2344 

10 

58 

20 

13 

4 

1.3516 

5.7031 

1 

53 

10 

Quarter  20 

7 

18 

1.0078 

9.5313 

2 

58 

20 

13 

4 

1.3516 

5.7031 

3 

102 

28 

21 

11 

1.3984 

2.3438 

4 

71 

10 

18 

10 

1.4922 

6.0156 

5 

68 

17 

8 

30 

1.2266 

5.8594 

6 

69 

10 

4 

21 

1.1328 

5.625 

7 

55 

7 

21 

16 

1.1719 

9.7656 

8 

86 

22 

24 

6 

0.5469 

5.0781 

9 

16 

24 

23 

30 

0.5313 

1.7188 

10 

106 

20 

16 

29 

1.3047 

2.0313 

Table  12.  Best  design  points  for  scenario  two 
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Design  points  that  rank  in  the  top  ten  for  both  quarters  are  in  bold  italics  in 
Table  11.  As  an  example,  if  it  was  determined  that  finishing  the  contingency  with  the 
highest  Ao  was  a  priority,  design  point  86  (Spares:  22,  Induction  Quantity:  24, 
Capacity:  6,  Degradation:  .5469,  Service  Time:  5.0781)  should  be  used  as  a  guide  to 
determine  the  optimal  settings  to  use  for  the  indicated  factors.  Design  point  86  shows  a 
high  number  of  spares  and  high  induction  quantity  provides  maximum  Ao.  For  a 
long-range  look  (20  quarters),  design  point  53  (Spares:  10,  Induction  Quantity:  7, 
Capacity:  18,  Degradation:  1.0078,  Service  Time:  9.5313)  should  be  used  as  a  guide  to 
determine  the  optimal  settings.  Design  point  53  shows  a  lower  number  of  spares,  lower 
induction  quantity  and  higher  capacity  provide  maximum  Ao  for  the  full  twenty  quarter 
period.  In  this  setting  (design  point  57)  increasing  the  depot  capacity  allows  a  higher  Ao 
while  maintaining  a  lower  level  of  spares  and  induction  quantity. 
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V.  CONCLUSIONS 


A.  RESEARCH  SUMMARY 

This  research  set  out  to  explore  the  capabilities  of  the  TLCM-AT.  Through 
development  of  a  Java  program  to  automate  implementation  of  DOE  with  TLCM-AT, 
this  thesis  produces  a  thorough  experimental  concept  and  analytical  technique  to  explore 
the  capabilities  TLCM-AT.  The  simulated  scenarios  used  in  this  analysis  fonn  a  strong 
foundation  for  further  studies  of  TLCM-AT,  using  DOE  to  analyze  life  cycles.  The 
analysis  produces  a  process  for  decision  makers  to  gain  insight  on  weapon  system  life 
cycles  using  TLCM-AT.  This  thesis  provides  the  following: 

■  A  framework  for  using  DOE  with  TLCM-AT. 

■  A  structured  use  of  TLCM-AT  for  USMC  decision  makers. 

■  Enhancements  that  enable  more  effective  usage  of  TLCM-AT. 

B.  DOE  AND  TLCM-AT 

TLCM-AT  uses  an  Access  database  for  all  inputs  and  outputs.  The  Java  program 
written  for  this  study  enables  manipulation  of  the  database  and  variation  of  input.  The 
program  development  applies  Java-Access  interaction  to  change  the  desired  elements  in 
the  database.  This  type  of  analysis  required  an  intimate  familiarity  with  TLCM-AT 
modeling  logic.  For  example,  changing  induction  quantity  requires  an  update  to  a  single 
input  table;  however,  implementing  a  change  in  the  number  of  spare  parts  requires 
updates  to  multiple  input  tables. 

When  performing  DOE  with  TLCM-AT,  the  Java  program  is  essential  to 
automating  the  process  of  executing  multiple  design  point  simulations.  The  program  is 
necessary  to  not  only  execute  changes  to  the  database,  but  to  also  keep  track  of  the 
specific  factor  settings  for  each  design  point  and  provide  the  output  in  a  fonn  amenable  to 
post  processing. 
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C.  TLCM-AT  OBSERVATIONS 

Several  observations  were  made  during  the  course  of  this  research  on  effective 
use  of  TLCM-AT.  Chapter  IV  shows  how  TLCM-AT  provides  a  quick-turn  analysis  of 
specific  “what-if  ’  scenarios.  This  is  accomplished  mainly  through  use  of  TLCM-AT ’s 
GUI  and  embedded  analytical  and  graphing  utilities.  The  GUI  contains  a  scenario  editor 
interface  to  allow  for  changes  in  the  following  areas: 

■  Capacity  constraints 

■  Component  induction 

■  Depot  upgrades 

■  Evacuation  probabilities 

■  Induction  schedules 

■  Limits  and  screens 

■  Logistical  consequences 

■  Maintenance  task  times 

■  Operational  tempo 

■  Parts  configuration 

■  Serial  number  planned  usage 

■  Shipment  times 

■  Spares  management 

■  Unscheduled  removal  rates 

TLCM-AT  is  a  data-driven  tool.  The  results  from  scenario  runs  are  completely 
dependent  on  the  quality  of  the  data  collected  and  modeled  in  the  database  such  as  miles 
driven,  shots  fired,  repair  times,  and  depot  capacities.  This  fact  demands  accurate 
recording  of  usage  data  throughout  the  fleet  enterprise  of  all  weapons  systems.  To  ensure 
valid  results,  recorded  data  must  be  accurate,  timely,  and  modeled  correctly.  The 
modeling  process  requires  a  solid  understanding  of  TLCM-AT  data  entry  to  represent 
specific  scenarios. 
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The  analysis  uses  the  average  Ao  taken  from  each  design  point.  Access  to  raw 
data  generated  from  each  replication,  within  each  design  point,  is  not  available  for 
analysis.  Access  to  TLCM-AT  raw  output  data  would  allow  mapping  into  distributions 
and  analysis  of  any  outliers. 

The  process  of  conducting  many  simulation  runs  would  be  greatly  enhanced  if 
TLCM-AT  could  be  run  using  a  cluster  of  computers.  This  approach  was  unsuccessful 
due  to  the  inability  for  TLCM-AT  to  be  run  in  a  headless  mode  (without  a  GUI).  The 
NPS  SEED  Center  computer  cluster  contains  compute  nodes  without  monitors  to  display 
the  dialogue  boxes  required  by  TLCM-AT. 

The  final  lesson  learned  concerns  the  extensive  knowledge  of  the  TLCM-AT 
modeling  process  required  to  simulate  different  scenarios.  The  scenarios  were  modeled 
by  a  modeler  and  analyst  for  Clockwork  Solutions  (Dr.  Peter  Figliozzi).  While  a  basic 
knowledge  of  TLCM-AT ’s  functional  utilities  will  allow  editing  of  various  inputs 
through  the  GUI,  a  more  robust  knowledge  is  essential  when  modeling  different  scenarios 
via  the  database.  Scenario  development  should  be  accessible  to  the  typical  user. 

D.  FOLLOW-ON  RESEARCH 

This  thesis  serves  as  a  template  for  future  life-cycle  studies  utilizing  TLCM-AT 
and  DOE.  Real-life  scenarios,  or  COAs,  can  be  modeled  and  many  different  MOEs  can 
be  studied.  A  list  of  possible  MOEs  to  study  in  the  future  is  shown  in  Table  13. 
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Measure  of  Effectiveness/Performance 

TLCM-AT  Output  Table 

Description 

Age  of  system 

out  Age  by  type 

Age  is  the  average  hours  accumulated  divided  by  average  number  of  objects  in  use. 

Unscheduled  maintenance  events 

out  Aircraft  events 

Events  include  part  failures,  life  limited,  induction  and  repairs. 

Awaiting  maintenance  (AWM)  status 

out  AWM 

Time  each  part  spent  in  a  waiting  queue,  such  a  AWM  status,  request  and  total  repairs. 

AWM  times  and  probablities 

out  AWM  times  and  probabilities 

Time  in  AWM,  probability  towards  availability  and  total  number  of  requests. 

Awaiting  parts  (AWP)  status 

out  AWP  items 

Time  spent  in  waiting  queue;  includes  AWP  status,  requests  and  total  repairs. 

out  AWP  times  and  probabilities 

Time  in  AWP,  probability  towards  availability  and  total  number  of  requests. 

Number  of  backordered  items 

out  Backordered  items 

Average  number  of  requests  resulting  in  backorder. 

Number  of  condemnations  (type) 

out  Condemnations 

Number  of  condemnations  per  type  per  quarter. 

Number  of  condemnations  (depot) 

out  Depot  ENSIP  and  Condemnation 

Average  number  of  types  sent  and  condemned  to  depot  per  quarter. 

Time  spent  in  depot 

out  Depot  Flow  Time 

Average  time  spent  in  the  depot  by  type. 

Scheduled  vs.  Achieved  operating  hours 

out  Flying  Hours 

Scheduled  and  achieved  hours  per  type  per  base  per  quarter. 

Logistics  response  delay  (LRT) 

out  Logistics  Response  Delay 

Average  LRT  between  bases. 

Number  of  LRU  removals 

out  LRU  events 

Average  LRU  removals  due  to  life  limits  and  failures  by  base  by  quarter. 

Number  of  new  buys 

out  New  Buys 

Average  number  of  new  buys  by  type,  base  and  quarter. 

Number  of  types  reaching  0-limit  screen 

out  Reaching  O-screen  items 

Average  of  types  reaching  O-limit  screens  by  base  by  quarter. 

Number  of  shipments 

out  Shipments 

Average  number  of  shipments  between  bases  by  type  by  quarter. 

Number  of  maintenance  tasks  performed 

out  Task  Performed 

Average  number  of  maintenance  tasks  performed  by  type  by  base  by  quarter. 

Number  of  unserviceable  parts 

out  Uninstalled-Servicable  items 

Average  number  of  unservicable  types  by  depot  by  week. 

Table  13.  MOEs  for  future  study 


It  must  be  noted  that  varying  factors  other  than  those  in  this  thesis  or  gathering 
different  outputs  will  require  additional  Java  coding  similar  to  the  code  detailed  in 
Lieutenant  Commander  Garcia’s  thesis  (to  be  completed  in  September  2008). 

Future  work  could  also  entail  a  study  to  integrate  the  cluster  of  computers  into  the 
TLCM-AT  and  DOE  process.  This  process  will  require  a  joint  effort  between 
Clockwork  Solutions  and  SEED  Center  research  personnel. 
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APPENDIX 


low  level 

1 

1 

1 

0.5 

0 

1 

1 

1 

0.5 

0 

high  level 

30 

30 

30 

1.5 

10 

30 

30 

30 

1.5 

10 

decimals 

0 

0 

0 

4 

4 

0 

0 

0 

4 

4 

factor  name 

Spares 

IQ 

1  Cap 

Deg 

ST 

Spares 

IQ 

1  Cap 

Deg 

ST 

8 

14 

12 

0.9531 

3.3594 

3 

11 

10 

0.8359 

5.4688 

27 

10 

13 

0.9609 

0.9375 

26 

15 

5 

0.5938 

5.3906 

14 

23 

1 

0.7734 

4.1406 

11 

28 

3 

0.9141 

7.2656 

21 

27 

10 

0.8672 

4.375 

16 

26 

6 

0.9375 

6.3281 

1 

12 

17 

0.7344 

1 .01 56 

3 

10 

20 

0.6016 

7.6563 

21 

13 

21 

0.5078 

3.9844 

25 

5 

16 

0.8984 

9.9219 

12 

30 

23 

0.7891 

1 .5625 

10 

28 

28 

0.6563 

7.1094 

18 

21 

27 

0.5625 

3.4375 

26 

25 

26 

0.8438 

9.375 

2 

2 

8 

0.7031 

1 .9531 

11 

15 

2 

0.6953 

7.9688 

30 

3 

7 

0.7656 

1.1719 

25 

12 

13 

0.6172 

7.3438 

2 

29 

15 

0.7813 

3.125 

15 

20 

7 

0.8125 

7.1875 

28 

30 

8 

0.8828 

1 .4063 

19 

25 

5 

0.6406 

6.1719 

15 

8 

29 

0.6797 

4.0625 

7 

2 

16 

0.9063 

8.2031 

23 

7 

28 

0.8516 

0.8594 

26 

13 

19 

0.5859 

6.4844 

8 

16 

29 

0.7266 

4.6875 

2 

24 

20 

0.9688 

7.7344 

24 

23 

30 

0.5313 

1 .71 88 

18 

26 

25 

0.6719 

9.6875 

8 

6 

7 

1 .2422 

0.3906 

15 

13 

7 

1.4609 

7.4219 

29 

11 

9 

1.2813 

4.2188 

25 

3 

4 

1.0781 

7.8125 

6 

23 

9 

1 .0234 

2.1094 

13 

16 

10 

1.2891 

5.2344 

20 

29 

12 

1.25 

1.25 

28 

25 

5 

1.375 

7.5 

9 

7 

25 

1.4531 

4.9219 

10 

7 

18 

1.0078 

9.5313 

19 

9 

20 

1.4219 

3.75 

26 

4 

28 

1.125 

9.2188 

7 

22 

23 

1 .4766 

3.2813 

7 

21 

16 

1.1719 

9.7656 

22 

19 

29 

1 .4844 

1.875 

27 

26 

25 

1.3125 

9.8438 

9 

8 

14 

1.1094 

3.2031 

6 

4 

11 

1.1797 

6.0938 

28 

1 

14 

1.0703 

1.4844 

20 

13 

4 

1.3516 

5.7031 

8 

22 

9 

1.5 

0.5469 

4 

25 

13 

1.4453 

10 

30 

28 

12 

1.2031 

1.6406 

18 

20 

14 

1.3359 

7.0313 

9 

14 

23 

1.0547 

3.5938 

13 

11 

27 

1.1406 

5.5469 

19 

14 

30 

1.1953 

4.8438 

17 

4 

18 

1.0156 

6.9531 

14 

22 

22 

1 .2578 

0.7031 

11 

18 

25 

1.4297 

7.5781 

17 

19 

28 

1.3906 

1 .3281 

27 

17 

20 

1.3672 

8.9063 

43 


low  level 

1 

1 

1 

0.5 

0 

1 

1 

1 

0.5 

0 

high  level 

30 

30 

30 

1.5 

10 

30 

30 

30 

1.5 

10 

decimals 

0 

0 

0 

4 

4 

0 

0 

0 

4 

4 

factor  name 

Spares 

IQ 

1  Cap 

Deg 

ST 

Spares 

IQ 

1  Cap 

Deg 

ST 

16 

16 

16 

1 

5 

14 

12 

3 

0.6094 

8.6719 

23 

17 

19 

1.0469 

6.6406 

28 

20 

21 

1.1641 

4.5313 

4 

21 

18 

1.0391 

9.0625 

5 

16 

26 

1.4063 

4.6094 

17 

8 

30 

1 .2266 

5.8594 

20 

3 

28 

1.0859 

2.7344 

10 

4 

21 

1.1328 

5.625 

15 

5 

25 

1.0625 

3.6719 

30 

19 

14 

1 .2656 

8.9844 

28 

21 

11 

1.3984 

2.3438 

10 

18 

10 

1 .4922 

6.0156 

6 

26 

15 

1.1016 

0.0781 

19 

1 

8 

1.2109 

8.4375 

21 

3 

3 

1.3438 

2.8906 

13 

10 

4 

1 .4375 

6.5625 

5 

6 

5 

1.1563 

0.625 

29 

29 

23 

1.2969 

8.0469 

20 

16 

29 

1.3047 

2.0313 

1 

28 

24 

1 .2344 

8.8281 

6 

19 

18 

1.3828 

2.6563 

29 

2 

16 

1.2188 

6.875 

16 

11 

24 

1.1875 

2.8125 

3 

1 

23 

1.1172 

8.5938 

12 

6 

26 

1.3594 

3.8281 

16 

23 

2 

1.3203 

5.9375 

24 

29 

15 

1.0938 

1.7969 

8 

24 

3 

1.1484 

9.1406 

5 

18 

12 

1.4141 

3.5156 

23 

15 

2 

1 .2734 

5.3125 

29 

7 

11 

1.0313 

2.2656 

7 

8 

1 

1.4688 

8.2813 

13 

5 

6 

1.3281 

0.3125 

23 

25 

24 

0.7578 

9.6094 

16 

18 

24 

0.5391 

2.5781 

2 

20 

22 

0.7188 

5.7813 

6 

28 

27 

0.9219 

2.1875 

25 

8 

22 

0.9766 

7.8906 

18 

15 

21 

0.7109 

4.7656 

11 

2 

19 

0.75 

8.75 

3 

6 

26 

0.625 

2.5 

22 

24 

6 

0.5469 

5.0781 

21 

24 

13 

0.9922 

0.4688 

12 

22 

11 

0.5781 

6.25 

5 

27 

3 

0.875 

0.7813 

24 

9 

8 

0.5234 

6.7188 

24 

10 

15 

0.8281 

0.2344 

9 

12 

2 

0.5156 

8.125 

4 

5 

6 

0.6875 

0.1563 

22 

23 

17 

0.8906 

6.7969 

25 

27 

20 

0.8203 

3.9063 

3 

30 

17 

0.9297 

8.5156 

11 

18 

27 

0.6484 

4.2969 

23 

9 

22 

0.5 

9.4531 

27 

6 

18 

0.5547 

0 
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